Search Results: "Colin Watson"

26 February 2012

Gregor Herrmann: RC bugs 2012/08

here comes my usual report about my work on RC bugs. this time a bit more "added a comment" or "sent a patch" but also some uploads.

30 January 2012

Colin Watson: APT resolver bugs

I've managed to go for eleven years working on Debian and nearly eight on Ubuntu without ever needing to teach myself how APT's resolver works. I get the impression that there's a certain mystique about it in general (alternatively, I'm just the last person to figure this out). Recently, though, I had a couple of Ubuntu upgrade bugs to fix that turned out to be bugs in the resolver, and I thought it might be interesting to walk through the process of fixing them based on the Debug::pkgProblemResolver=true log files. Breakage with Breaks The first was Ubuntu bug #922485 (apt.log). To understand the log, you first need to know that APT makes up to ten passes of the resolver to attempt to fix broken dependencies by upgrading, removing, or holding back packages; if there are still broken packages after this point, it's generally because it's got itself stuck in some kind of loop, and it bails out rather than carrying on forever. The current pass number is shown in each "Investigating" log entry, so they start with "Investigating (0)" and carry on up to at most "Investigating (9)". Any packages that you see still being investigated on the tenth pass are probably something to do with whatever's going wrong. In this case, most packages have been resolved by the end of the fourth pass, but xserver-xorg-core is causing some trouble. (Not a particular surprise, as it's an important package with lots of relationships.) We can see that each breakage is:
Broken xserver-xorg-core:i386 Breaks on xserver-xorg-video-6 [ i386 ] < none > ( none )
This is a Breaks (a relatively new package relationship type introduced a few years ago as a sort of weaker form of Conflicts) on a virtual package, which means that in order to unpack xserver-xorg-core each package that provides xserver-xorg-video-6 must be deconfigured. Much like Conflicts, APT responds to this by upgrading providing packages to versions that don't provide the offending virtual package if it can, and otherwise removing them. We can see it doing just that in the log (some lines omitted):
Investigating (0) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-tseng:i386
Investigating (1) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-i740:i386
Investigating (2) xserver-xorg-core [ i386 ] < 2:1.7.6-2ubuntu7.10 -> 2:1.11.3-0ubuntu8 > ( x11 )
  Fixing xserver-xorg-core:i386 via remove of xserver-xorg-video-nv:i386
OK, so that makes sense - presumably upgrading those packages didn't help at the time. But look at the pass numbers. Rather than just fixing all the packages that provide xserver-xorg-video-6 in a single pass, which it would be perfectly able to do, it only fixes one per pass. This means that if a package Breaks a virtual package which is provided by more than ten installed packages, the resolver will fail to handle that situation. On inspection of the code, this was being handled correctly for Conflicts by carrying on through the list of possible targets for the dependency relation in that case, but apparently when Breaks support was implemented in APT this case was overlooked. The fix is to carry on through the list of possible targets for any "negative" dependency relation, not just Conflicts, and I've filed a patch as Debian bug #657695. My cup overfloweth The second bug I looked at was Ubuntu bug #917173 (apt.log). Just as in the previous case, we can see the resolver "running out of time" by reaching the end of the tenth pass with some dependencies still broken. This one is a lot less obvious, though. The last few entries clearly indicate that the resolver is stuck in a loop:
Investigating (8) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (8) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
Investigating (9) dpkg [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( admin )
Broken dpkg:i386 Breaks on dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils ) (< 1.15.8)
  Considering dpkg-dev:i386 29 as a solution to dpkg:i386 7205
  Upgrading dpkg-dev:i386 due to Breaks field in dpkg:i386
Investigating (9) dpkg-dev [ i386 ] < 1.15.5.6ubuntu4.5 -> 1.16.1.2ubuntu5 > ( utils )
Broken dpkg-dev:i386 Depends on libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl ) (= 1.16.1.2ubuntu5)
  Considering libdpkg-perl:i386 12 as a solution to dpkg-dev:i386 29
  Holding Back dpkg-dev:i386 rather than change libdpkg-perl:i386
The new version of dpkg requires upgrading dpkg-dev, but it can't because of something wrong with libdpkg-perl. Following the breadcrumb trail back through the log, we find:
Investigating (1) libdpkg-perl [ i386 ] < none -> 1.16.1.2ubuntu5 > ( perl )
Broken libdpkg-perl:i386 Depends on perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
  Considering perl:i386 1472 as a solution to libdpkg-perl:i386 12
  Holding Back libdpkg-perl:i386 rather than change perl:i386
Investigating (1) perl [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl:i386 Depends on perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl ) (= 5.14.2-6ubuntu1)
  Considering perl-base:i386 5806 as a solution to perl:i386 1472
  Removing perl:i386 rather than change perl-base:i386
Investigating (1) perl-base [ i386 ] < 5.10.1-8ubuntu2.1 -> 5.14.2-6ubuntu1 > ( perl )
Broken perl-base:i386 PreDepends on libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (>= 2.11)
  Considering libc6:i386 -17473 as a solution to perl-base:i386 5806
  Added libc6:i386 to the remove list
Investigating (0) libc6 [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs )
Broken libc6:i386 Depends on libc-bin [ i386 ] < 2.11.1-0ubuntu7.8 -> 2.13-24ubuntu2 > ( libs ) (= 2.11.1-0ubuntu7.8)
  Considering libc-bin:i386 10358 as a solution to libc6:i386 -17473
  Removing libc6:i386 rather than change libc-bin:i386
So ultimately the problem is something to do with libc6; but what? As Steve Langasek said in the bug, libc6's dependencies have been very carefully structured, and surely we would have seen some hint of it elsewhere if they were wrong. At this point ideally I wanted to break out GDB or at the very least experiment a bit with apt-get, but due to some tedious local problems I hadn't been able to restore the apt-clone state file for this bug onto my system so that I could attack it directly. So I fell back on the last refuge of the frustrated debugger and sat and thought about it for a bit. Eventually I noticed something. The numbers after the package names in the third line of each of these log entries are "scores": roughly, the more important a package is, the higher its score should be. The function that calculates these is pkgProblemResolver::MakeScores() in apt-pkg/algorithms.cc. Reading this, I noticed that the various values added up to make each score are almost all provably positive, for example:
         Scores[I->ID] += abs(OldScores[D.ParentPkg()->ID]);
The only exceptions are an initial -1 or -2 points for Priority: optional or Priority: extra packages respectively, or some values that could theoretically be configured to be negative but weren't in this case. OK. So how come libc6 has such a huge negative score of -17473, when one would normally expect it to be an extremely powerful package with a large positive score? Oh. This is computer programming, not mathematics ... and each score is stored in a signed short, so in a sufficiently large upgrade all those bonus points add up to something larger than 32767 and everything goes haywire. Bingo. Make it an int instead - the number of installed packages is going to be on the order of tens of thousands at most, so it's not as though it'll make a substantial difference to the amount of memory used - and chances are everything will be fine. I've filed a patch as Debian bug #657732. I'd expected this to be a pretty challenging pair of bugs. While I certainly haven't lost any respect for the APT maintainers for dealing with this stuff regularly, it wasn't as bad as I thought. I'd expected to have to figure out how to retune some slightly out-of-balance heuristics and not really know whether I'd broken anything else in the process; but in the end both patches were very straightforward.

16 January 2012

Rapha&#235;l Hertzog: Review of my Debian related goals for 2011

Last year I shared my Debian related goals for 2011 . I tend to put more goals than what I can reasonably complete alone and this year was no exception. Let s have a look.
  1. Translate my Debian book into English: PARTLY DONE
    It took more time than expected to prepare and to run the fundraising campaign but it has been successful and the translation is happening right now.
  2. Finish multiarch support in dpkg: DONE BUT NOT ENTIRELY MERGED YET
    Yes, multiarch support was already in the pipe last year in January. I completed the development between January and April (it was sponsored by Linaro) and since then it has mostly been waiting on Guillem to review it, tweak it, and integrate it.
  3. Make deb files use XZ compression by default: TRIED BUT ABANDONED
    After discussing the issue with Colin Watson and Joey Hess during debconf, I came to the conclusion that it was not really desirable at this point. The objections were that debian-installer was not ready for it and that it adds a new dependency on xz for debootstrap to work on non-Debian systems. I believe that the debian-installer side is no longer a problem since unxz is built in busybox-udeb (since version 1:1.19.3-2). For the other side, there s not much to do except ensuring that xz is portable to all the other OS we care about. DAK has been updated too (see #556407).
  4. Be more reactive to review/merge dpkg patches: PARTLY DONE
    I don t think we had any patch that received zero input. We still have a backlog of patches, and the situation is far from ideal but the situation improved.
  5. Implement the rolling distribution proposed as part of the CUT project and try to improve the release process: NOT DONE
    We had a BoF during debconf, we discussed it at length on debian-devel, but in the end we did nothing out of it. Except Josselin Mouette who wrote a proof of concept for his idea. For me testing is already what people are expecting from a rolling distribution. It s just a matter of documenting how to effectively use testing, and of some marketing by defining rolling as alias to testing.
  6. Work more regularly on the developers-reference: PARTLY DONE
    I did contribute some new material to the document but not as much as I could have hoped. On the other hand, I have been rather reactive to ensure that sane patches got merged. We need more people writing new parts and updating the existing content.
  7. Write a 10-lesson course called Smart Package Management : NOT DONE
  8. Create an information product (most likely an ebook or an online training) and sell it on my blog: NOT DONE
    This was supposed to happen after the translation of the Debian Administrator s Handbook. Since the translation is not yet over, I did not start to work on this yet.
  9. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products: NOT REACHED
    My target was rather aggressive with 700 each month, and given that I did not manage to complete any information product, I m already very pleased to have achieved a mean amount of 204 of donations each month (min: 91 , max: 364 ). It s more than two times better than in 2010. Thank you! Note that those figures do not take into account the revenues of the fundraising of the Debian Administrator s Handbook since they will be used for its translation.
That makes quite a lot of red (for things that I did not achieve) on the other hand I completed projects that I did not foresee and did not plan. For instance improving dpkg-buildflags and then merging Kees Cook work on hardened build flags was an important step for Debian. This was waiting for so long already

2 comments Liked this article? Click here. My blog is Flattr-enabled.

15 January 2012

Gregor Herrmann: RC bugs 2012/02

while working on RC bugs, I noticed that some DDs sent patches to the BTS in the last weeks (yeah!) but did not upload the fixes in NMUs. short request: could you please either state the reason for not uploading, or just go ahead and dput/dupload the fixed packages?

& here's my list:

10 December 2011

Tim Retout: SFTP default umask

So I was about to configure an FTP server to let a friend upload content for a website... and then I came to my senses and remembered sftp exists. It's supported by the same graphical clients, and avoids me having to figure out SSL certificates and so on. Next problem: we want to both edit the site. Okay, so I create a group, make it the default group for both users... and now I need to set the umask to 002 so that all group members can edit all files. There's no option in the client... Skimming Debian bug #496843 (closed Apr 2010, thanks Colin Watson!) we can set this in sshd_config these days - no need to mess about with wrapper scripts. Very easy:
Subsystem sftp /usr/lib/openssh/sftp-server -u 002
Now all content created through the sftp client is group-writable, and owned by the default group of each user! See 'man (8) sftp-server'.

5 December 2011

Ana Beatriz Guerrero Lopez: RCBSaturday

On Saturday evening I started talking with M nica about Bug Squashing Parties and how they work. I am not sure how it happened, we started doing one. Then it was too fun to stop :P NMUs by M nica: Funny, there were a couple of NMUs by others uploading patchs by M nica. NMUs by Ana: I also closed #646449 (libosip2) that has been fixed in a new release upload, sponsored a NMU to Sven Joachim who had a patch sitting in the BTS for a month NMU-fixing #646147 (lie), sponsored dbus-c++ and reviewed for sponsoring a new version inspircd.

2 December 2011

Rapha&#235;l Hertzog: My Debian Activities in November 2011

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (310.73 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg: Multi-Arch Saga I know lots of people are waiting the landing of multiarch in Debian unstable, and so am I. Things are progressing, though not as quickly as I hoped. Guillem merged about half of the branch between the 24th October and the 6th of November. After that most of the work happened on his personal repository in his pu/multiarch/master branch. I verify this repository from time to time because Guillem does not inform me when he has made progress. I noticed changes on his repository on the 10th, 19th, 23th, 28th of November and on the 1th of December. He announced a long time ago that he had some interface changes and up to now only wrote about the switch from the command-line option --foreign-architecture (to put in /etc/dpkg/dpkg.cfg) to the explicit command dpkg --add-architecture that only needs to be called once (see mail here). As of today (December 2th), the promised email for the other interface changes is still not here. On November 23th, I reviewed Guillem s work and tried to run the code in his branch. I spent the whole day chasing up regressions and submitted lots of fixes to Guillem. Thanks to the extensive test-suite I wrote when I developed my branch, it has been fairly easy to track them all down. All the issues I reported have been fixed in the latest version of Guillem s branch although the fixes are often slightly different from those that I submitted. Dpkg: Squeeze Backport At the start of the month, I uploaded what I expected to be a fairly uncontroversial backport of dpkg 1.16.1.1. It turns out I was wrong. After some discussion, I think we came to an agreement that it was acceptable to backport dpkg-dev and libdpkg-perl only. My goal was not to bring the latest dpkg to users but to make it easier for package maintainers to backport packages using new features provided by dpkg-dev >= 1.16 (such as hardening build flags, the makefile snippets provided in /usr/share/dpkg/, or the improved dpkg-buildflags interface). Thus I modified the source package uploaded to squeeze-backports to build only dpkg-dev and libdpkg-perl. It has been uploaded on November 23th and it s waiting in the NEW queue for a backports admin to process it. Misc Dpkg Work I merged a patch of Colin Watson to be able to verify build-dependencies for a foreign architecture (taking into account the Multi-Arch status of each package listed). I released dpkg 1.16.1.2 with two minor fixes that were sitting in the sid branch. I wanted to get rid of this so that the path is clear for a 1.16.2 upload with multiarch. The package just migrated to testing so we re fine. I spent another day doing dpkg bug triaging on Launchpad, we re now down to 77 bugs with many of them tagged as incomplete and likely to expire in 2 months. The Debian Administrator s Handbook eBookWe released a sample chapter so that it s easier to have an idea of the quality of the book. The chapter covers the APT tools quite extensively. I bet that even you could learn something about apt-get/aptitude The crowfunding campaign on Ulule ended on November 28th.
With 675 supporters, we raised 24385 EUR. Of those, 14935 EUR have been put in the liberation fund and the rest corresponds to the various pre-orders and rewards offered. This means that the translation will happen (we just started) but that the book is currently not going to be released under a free license. Don t despair As planned, the liberation campaign is carried on until the 25 K target is reached! Instead of being hosted on Ulule, this permanent campaign is on the project website at debian-handbook.info/liberation/. Note that any contribution of 10 EUR or more means that you get a copy of the ebook as soon as it s available (even if the liberation target is not reached). Package Tracking System At the start of the month, I filed two ideas of improvements for the PTS in the bug tracking system: #647258 is about showing outstanding bugs that relate to a release goal and #647901 is about warning maintainers that the package is affected by a current transition. If you re a coder and want to start contributing to Debian and its QA team, those bugs could be interesting targets for a start. :-) In both cases, I have been in contact with members of the release team because those ideas require some structured data from the release team as input. Thanks to Meddi Dohguy and Niels Thykier for their help. Later in the month, the topic of relocating the PTS once again came up. For historical reasons, the PTS was hosted on master.debian.org together with the BTS. Nowadays the BTS has its own host and it made no sense anymore to have the PTS separate from the rest of the QA services hosted on qa.debian.org (currently quantz.debian.org). So together with Martin Zobel Helas we took care to plan the migration and on November 19th we executed the plan. It worked like a charm and almost nobody noticed (only one undocumented dependency was missed, which broke the SOAP interface). Misc Packaging Work WordPress was broken in Ubuntu and it was also not properly synchronized with Debian due to an almost useless change on their side. Thus I requested a sync so that the working version from Debian gets imported in Ubuntu. I sponsored the docbook-xsl 1.76.1 upload that I needed for Publican. Then I updated Publican just to discover that the test-suite triggers a new bug in fop (filed as #649476). I disabled the test-suite temporarily and uploaded Publican 2.8 to unstable. BTW, I also filed 2 upstream bugs with patches for issues I discovered while trying to generate the sample chapter of my book (see here and here). I uploaded a version 0.7.1 of nautilus-dropbox and fixed #648215 at the same time. I made an NMU of bison to fix a long-standing release critical bug that hit me once more during an upgrade (see #645038). I uploaded to experimental a new version of gnome-shell-timer compatible with GNOME 3.2. I took the opportunity to install from experimental the few GNOME 3.2 packages which are not yet in unstable Thanks See you next month for a new summary of my activities.

7 comments Liked this article? Click here. My blog is Flattr-enabled.

27 November 2011

Gregor Herrmann: RC bugs 2011/47

after the rather intensive last week, this week was more business as usual. here's the list:

20 November 2011

Gregor Herrmann: RC bugs 2011/46

as expected this week was characterised by fixing bugs around the perl 5.14 transition. besides that I've also uploaded a couple of NMUs with patches kindly provided in the BTS by some tireless bug fixers.

6 November 2011

Gregor Herrmann: RC bugs 2011/44

this week showed an interesting mixture of RC bug fixes: applying patches from others, writing one on my one, adjusting build dependencies, & also closing some bugs that did not occur anymore during the last Grid'5000 run & are also unreproducible locally (thanks to Lucas for the hint).

24 October 2011

Colin Watson: Quality in Ubuntu 12.04 LTS

As is natural for an LTS cycle, lots of people are thinking and talking about work focused on quality rather than features. With Canonical extending LTS support to five years on the desktop for 12.04, much of this is quite rightly focused on the desktop. I'm really not a desktop hacker in any way, shape, or form, though. I spent my first few years in Ubuntu working mainly on the installer - I still do, although I do some other things now too - and I used to say only half-jokingly that my job was done once X started. Of course there are plenty of bugs I can fix, but I wanted to see if I could do something with a bit more structure, so I got to thinking about projects we could work on at the foundations level that would make a big difference. Image build pipeline One difficulty we have is that quite a few of our bugs - especially installer bugs, although this goes for some other things too - are only really caught when people are doing coordinated image testing just before a milestone release. Now, it takes a while to do all the builds and then it takes a while to test them. The excellent work of the QA team has meant that testing is much quicker now than it used to be, and a certain amount of smoke-testing is automated (particularly for server images). On the other hand, the build phase has only got longer as we've added more flavours and architectures, particularly as some parts of the process are still serialised per architecture or subarchitecture so ARM builds in particular take a very long time indeed. Exact timings are a bit difficult to get for various reasons, but I think the minimum time between a developer uploading a fix and us having a full set of candidate images on all architectures including that fix is currently somewhere north of eight hours, and that's with people cutting corners and pulling strings which is a suboptimal thing to have to do around release time. This obviously makes us reluctant to respin for anything short of showstopper bugs. If we could get things down to something closer to two hours, respins would be a much less horrible proposition and so we might be able to fix a few bugs that are serious but not showstoppers, not to mention that the release team would feel less burned out. We discussed this problem at the release sprint, and came up with a laundry list of improvements; I've scheduled this for discussion at UDS in case we can think of any more. Please come along if you're interested! One thing in particular that I'm working on is refactoring Germinate, a tool which dates right back to our first meeting before Ubuntu was even called Ubuntu and whose job is to expand dependencies starting from our lists of "seed" packages; we use this, among other things, to generate Task fields in the archive and to decide which packages to copy into our images. This was acceptably quick in 2004, but now that we run it forty times (eight flavours multiplied by five architectures) at the end of every publisher run it's actually become rather a serious performance problem: cron.germinate takes about ten minutes, which is over a third of the typical publisher runtime. It parses Packages files eight times as often as it needs to, Sources files forty times as often as it needs to, and recalculates the dependency tree of the base system five times as often as it needs to. I am confident that we can significantly reduce the runtime here, and I think there's some hope that we might be able to move the publisher back to a 30-minute cycle, which would increase the velocity of Ubuntu development in general. Maintaining the development release Our release cycle always starts with syncing and merging packages from Debian unstable (or testing in the case of LTS cycles). The vast majority of packages in Ubuntu arrive this way, and generally speaking if we didn't do this we would fall behind in ways that would be difficult to recover from later. However, this does mean that we get a "big bang" of changes at the start of the cycle, and it takes a while for the archive to be usable again. Furthermore, even once we've taken care of this, we have a long-established rhythm where the first part of the cycle is mainly about feature development and the second part of the cycle is mainly about stabilisation. As a result, we've got used to the archive being fairly broken for the first few months, and we even tell people that they shouldn't expect things to work reliably until somewhere approaching beta. This makes some kind of sense from the inside. But how are you supposed to do feature development that relies on other things in the development release? In the first few years of Ubuntu, this question didn't matter very much. Nearly all the people doing serious feature development were themselves serious Ubuntu developers; they were capable of fixing problems in the development release as they went along, and while it got in their way a little bit it wasn't all that big a deal. Now, though, we have people focusing on things like Unity development, and we shouldn't assume that just because somebody is (say) an OpenGL expert or a window management expert that they should be able to recover from arbitrary failures in development release upgrades. One of the best things we could do to help the 12.04 desktop be more stable is to have the entire system be less unstable as we go along, so that developers further up the stack don't have to be distracted by things wobbling underneath them. Plus, it's just good software engineering to keep the basics working as you go along: it should always build, it should always install, it should always upgrade. Ubuntu is too big to do something like having everyone stop any time the build breaks, the way you might do in a smaller project, but we shouldn't let things slide for months either. I've been talking to Rick Spencer and the other Ubuntu engineering leads at Canonical about this. Canonical has a system of "rotations", where you can go off to another team for a while if you're in need of a change or want to branch out a bit; so I proposed that we allow our engineers to spend a month or two at a time on what I'm calling the +1 Maintenance Team, whose job is simply to keep the development release buildable, installable, and upgradeable at all times. Rick has been very receptive to this, and we're going to be running this as a trial throughout the 12.04 cycle, with probably about three people at a time. As well as being professional archive gardeners, these people will also work on developing infrastructure to help us keep better track of what we need to do. For instance, we could deploy better tools from Debian QA to help us track uninstallable packages, or we could enhance some of our many existing reports to have bug links and/or comment facilities, or we could spruce up the weather report; there are lots of things we could do to make our own lives easier. By 12.04, I would like, in no particular order: Of course, this overlaps to a certain extent with the kinds of things that the MOTU team have been doing for years, not to mention with what all developers should be doing to keep their own houses in reasonable order, and I'd like us to work together on this; we're trying to provide some extra hands here to make Ubuntu better for everyone, not take over! I would love this to be an opportunity to re-energise MOTU and bring some new people on board. I've registered a couple of blueprints (priorities, infrastructure) for discussion at UDS. These are deliberately open-ended skeleton sessions, and I'll try to make sure they're scheduled fairly early in the week, so that we have time for break-out sessions later on. If you're interested, please come along and give your feedback!

16 October 2011

Gregor Herrmann: RC bugs 2011/41

here's my list of recent RC bug fixes, again comprising bugs which already had a patch in the BTS:

9 October 2011

Gregor Herrmann: RC bugs 2011/40

another week, another list of fixed RC bugs, again most of them with patches from ubuntu. &, as zack said today, the next freeze is not really imminent but it's getter closer.

8 October 2011

Steve Langasek: Free as in flickers

Ubuntu has had an explicit goal for several cycles now to achieve flicker-free booting: smooth graphical boot all the way through from bootloader to the desktop, with no messy modeswitches or cuts between text and graphics mode on the console. This is no small task, to be sure; even getting flicker-free transitions between plymouth and X for the KMS-capable video cards was a handful for 10.04 LTS. So I was suitably impressed when in the course of testing for the upcoming Ubuntu 11.10 release I found that on a laptop with the nouveau driver, we had achieved 100% flicker-free boot all the way from grub to X. And then I was astonished to find the grub-kernel transition was flicker-free when using the binary nvidia driver too! This is in large part thanks to Colin Watson's work upstream implementing VBE support in GRUB2, and Andy Whitcroft's adding proper support for a vesafb fallback when no KMS video driver is found. The architecture of how this all hangs together is now documented in the Ubuntu wiki. The irony is that, while nouveau gives a perfect flicker-free boot, and the nvidia binary drivers give a boot with just one modeswitch, both of my Intel systems show two modeswitches on boot - one between grub and the kernel, another between the kernel and X. Something to work on cleaning up next month, I think...

6 October 2011

Colin Watson: Top ideas on Ubuntu Brainstorm (August 2011)

The Ubuntu Technical Board conducts a regular review of the most popular Ubuntu Brainstorm ideas (previous reviews conducted by Matt Zimmerman and Martin Pitt). This time it was my turn. Apologies for the late arrival of this review. Contact lens in the Unity Dash (#27584) Unity supports Lenses, which provide a consistent way for users to quickly search for information via the Dash. Current lenses include Applications, Files, and Music, but a number of people have asked for contacts to be accessible using the same interface. While Canonical's DX team isn't currently working on this for Ubuntu 11.10 or 12.04, we'd love somebody who's interested in this to get involved. Allison Randal explains how to get started, including some skeleton example code and several useful links. Displaying Ubuntu version information (#27460) Several people have asked for it to be more obvious what Ubuntu version they're running, as well as other general information about their system. John Lea, user experience architect on the Unity team, responds that in Ubuntu 11.10 the new LightDM greeter shows the Ubuntu version number, making that basic information very easily visible. For more detail, System Settings -> System Info provides a simple summary. Volume adjustments for headphone use (#27275) People often find that they need to adjust their sound volume when plugging in or removing headphones. It seems as though the computer ought to be able to remember this kind of thing and do it automatically; after all, a major goal of Ubuntu is to make the desktop Just Work. David Henningson, a member of Canonical's OEM Services group and an Ubuntu audio developer, responds on his blog with a summary of how PulseAudio jack detection has improved matters in Ubuntu 11.10, and what's left to do:
The good news: in the upcoming Ubuntu Oneiric (11.10), this is actually working. The bad news: it isn't working for everyone.
Making it easier to find software to handle a file (#28148) Ubuntu is not always as helpful as it could be when you don't have the right software installed to handle a particular file. Michael Vogt, one of the developers of the Ubuntu Software Center, responded to this. It seems that most of the pieces to make this work nicely are in place, but there are a few more bits of glue required:
Thanks a lot for this suggestion. I like the idea and it's something that software-center itself supports now. In the coming version 5.0 we will offer to "sort by top-rated" (based on the ratings&reviews data). It's also possible to search for an application based on its mime data. To search for a mime-type, you can enter "mime:text/html" or "mime:audio/ogg" into the search field. What is needed however is better integration into the file manager nautilus. I will make sure this gets attention at the next developer meeting and filed bug #860536 about it. In nautilus, there is now a button called "Find applications online" available as an option when opening an unknown file or when the user selects "open with...other application" in the context menu. But that will not use the data from software-center.
Show pop-up alert on low battery (#28037) Some users have reported on Brainstorm that they are not alerted frequently enough when their laptop's battery is low, as they clearly ought to be. This is an odd one, because there are already several power alert levels and this has been working well for us for some time. Nevertheless, enough people have voted for this idea that there must be something behind it, perhaps a bug that only affects certain systems. Martin Pitt, technical lead of the Ubuntu desktop team, has responded directly to the Brainstorm idea with a description of the current system and how to file a bug when it does not work as intended.

4 October 2011

Andreas Barth: Building a mipsel porter box and buildd

As grub is now working on the loongson 2e boxes as well (thanks to phcoder and Colin Watson), it is time to move the buildds running at my home to a data center (previously we couldn't remote manage the kernels / boot flags without VGA console, which means no data center usage). Also one of the boxes could be converted to a porter machine, so that we could get an mipsel porter box again. The machines are delivered only with 256M of ram, which is a bit too less for usage. Thanks to Zugschlus (Marc Haber) I got 1g ram for both machines going to Vienna (one buildd, one porter box), and thanks to Robert Grimm a 160g harddisk to replace the build-in 40g in the porter box (the buildd can cope with 40g fine). The additional ram modules and hard disk are visible on the following picture.
eysler.debian.org is now DSAed, online and building packages (including autosigning). I will shutdown monteverdi.ayous.org which is still at my place in the next days, reinstall the system and send it to Darmstadt as another buildd for data center hosting there. After that happens all mipsel buildds are DSAed as it should be, and are running in a data center and not via some DSL line. (In case you're looking for hardware at your place, there are a couple of loongson 2f-systems available to buy. 2f is the successor cpu of 2e. Some 2f systems get delivered with Debian installed on it, see e.g. http://www.aliexpress.com/product-gs/290632002-Yeeloong-8101-Laptop-10-inch-Debian-EU-Adaptor-version--wholesalers.html. However, for buildd usage, the 2e are fine as well, and we got them sponsored some time ago.)

2 October 2011

Gregor Herrmann: RC bugs 2011/39

another weekly report about RC bugs I've fixed, this time mostly by applying patches from ubuntu:

25 September 2011

Gregor Herrmann: RC bugs 2011/38

quick overview over my RC bug activities during the last week: random notes:

16 September 2011

Rapha&#235;l Hertzog: How to triage bugs in the Debian bug tracking system

Triaging bugs is one of the easiest way to start contributing to Debian. I ll teach you the basics in this article. 1. Prerequisites All interactions with the Debian Bug Tracking System (BTS) happen through email so you need to have an email account with an address that you re willing to make public. All the mail that you send to the BTS will be archived and publicly available through its web-interface. This also means that you should have some spam filters in place because it will inevitably be harvested by spammers. :-( To ensure that this email address is consistently used by the various tools that we re going to use, it s a good idea to put this email address in the DEBEMAIL environment variable. You can also specify your full name in DEBFULLNAME (in case you don t want to use the name associated with your Unix account). You usually do this by modifying ~/.bashrc (if you use bash as login shell):
export DEBEMAIL="hertzog@debian.org"
export DEBFULLNAME="Rapha l Hertzog"
You should also install the devscripts package, it provides the bts command that we re going to use. 2. Find a package or a team with too many bugs You can literally pick any popular software that s in Debian, they almost always get more bug reports than the maintainers can handle. Instead of picking a package, you can also select a packaging team and concentrate your efforts on the set of packages managed by the team. In any case, it s important to receive the bug traffic for the packages that you re going to work on. If you went for a specific package, you should subscribe to the package via the Package Tracking System (there s a subscribe box on the bottom left corner once you selected the source package of interest). If you decided to help a team, there s usually a dedicated mailing list receiving all bug traffic. You can browse a list of packages with the most bugs if you have troubles finding a package to work on. A stack of bug reports to triage 3. Triage bugs! Bug triaging is all about making sure that bugs are correctly classified so that when a developer looks at the bug list, he can quickly find bugs with all the information required to be able to fix them! 3.1 Adding information to bug Adding supplementary information is easily done just by sending a mail to XXXX@bugs.debian.org (replace XXXX with the bug number). But often you want to reply to a message in the bug history, in that case bts --mbox show XXXX is for you. It will grab the corresponding mailbox and open a mailer (mutt by default) on it. Now you can directly reply in your favorite mailer. 3.2 Classifying bugs The Debian BTS uses tags (click the link and read the doc!) to classify bugs. bts tag XXXX + foo will add the foo tag (replace the + with a to remove a tag). If you want to explain why you re adding a tag, you should instead reply in the bug log as explained above, put control@bugs.debian.org in Bcc (Blind Carbon Copy) and start the body of your message with your tag command:
tag XXXX + foo
thanks
But what tag should you add? When a bug is submitted, you should try to reproduce the bug. If you can reproduce it, then tag the bug confirmed (example in #641710). If you can t, you should request more information (ex: a sample document triggering the bug, a configuration file, the output of some relevant command, etc.) until you can reproduce it or conclude that it was a user mistake. When you request supplementary information due to this, you should tag the bug unreproducible moreinfo (example in #526774). moreinfo should be later dropped when the requested information are provided, and unreproducible should be dropped if those information were enough to actually help reproduce the bug (example in #526774). During that initial evaluation, it s also worth differentiating packaging bugs (which are specific to Debian) from upstream bugs (which are relevant also for non-Debian users). The latter should be tagged upstream (and forwarded upstream if the bug is reproducible or contains enough information for the upstream developers, example in #635112). If you saw a (viable) patch in the bug log, the bug should be tagged patch . This is usually done by the patch submitter but sometimes it s forgotten (example in #632460). Take care though to not reinstate the patch tag if it was initially set but then dropped by the package maintainer after a review of the patch. If the title of the bug report is not descriptive enough, you can change it with a retitle XXXX new-title command (example in #170850). You can also change the severity of the bug report depending on the impact of the problem (with a command severity XXXX new-severity , what a surprise!). Request for new features are wishlist , most documentation problems are minor . On the other side of the scale, you can use important for bugs that are very annoying but that should not block a release. serious , grave and critical are used for release critical bugs, check the official definitions of the severities (examples in 502738 or #506498). 3.3 Closing non-bugs and bugs that are already fixed If your analysis of the bug report is that it s not really a bug but a user mistake, then you should close it by sending a mail to XXXX-done@bugs.debian.org with some explanations of the user s mistake so that he can get past his problem (example in #592853). If the problem was a real bug, but one that is apparently already fixed, you should try to quickly find the version that fixed the bug. If you can t find it in the changelog (there s a link to it in the PTS, or you can use /usr/share/doc/package/changelog.Debian.gz), you ll make the safe assumption that the upstream version you re currently using is the first one where this is fixed. Then you send a mail to XXXX-done@bugs.debian.org but you start your mail with Version: version-that-fixed-the-bug and continue with a small explanation of why you believe the bug to be fixed by this version (example in #122948). 3.4 Reassigning misfiled bug reports Bug reports are not always filed against the proper package. Users file bugs against applications where they experience the bugs, but the real bug might be in an underlying library or application. When that happens, you should use the reassign XXXX correct-package version command to get it filed against the correct package. The version parameter is optional but should be provided if possible, it should be the oldest version that we know to have the problem (example in #626232). 3.5 Forwarding bugs Forwarding bugs means opening bug reports in the upstream bug tracker for issues that have been reported in Debian but that applies to the upstream (unmodified) source code. Be sure to include all the relevant information and a link to the corresponding Debian bug. Depending on the upstream bug tracker, you might have to open an account to be able to file new bug reports. On the Debian side, you must record that a bug has been forwarded with bts forwarded XXXX upstream-bug-url . upstream-bug-url is the URL corresponding to the upstream bug report you created (ex: http://projects.ciarang.com/p/feed2omb/issues/21/ recorded in #609345 ). If the upstream authors fix the bug you reported, you can tag the Debian bug with fixed-upstream so that it s easier to find bugs to close when the next upstream release comes out (example in #637275). 3.6 Updating version information The Debian BTS uses version tracking to know which package versions are affected by a given bug. It s particularly important to have correct version information for release critical bugs since it might affect the migration of packages to testing. You can learn more on this topic here: http://wiki.debian.org/HowtoUseBTS. 4. More advice Colin Watson wrote a constructive rant explaining some mistakes that bug triagers are often doing. While it refers mainly to Ubuntu s launchpad, the advice apply equally as well to Debian. Check it out to become a better bug triager! Note that you can refer to this article with this shorter URL: http://raphaelhertzog.com/go/bugtriaging/

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

One comment Liked this article? Click here. My blog is Flattr-enabled.

21 July 2011

Rapha&#235;l Hertzog: People behind Debian: Martin Michlmayr, former Debian Project Leader

Martin Michlmayr is a Debian developer since 2000 and I share quite a few things with him, starting with his age and involvement in the quality assurance team. He managed to be elected Debian Project Leader in 2003 and 2004. He s no longer as active as he used to be but his input is always very valuable and he continues to do very interesting things in particular concerning the support of NAS devices. Read on for the details. Raphael: Who are you? Martin: I m Martin Michlmayr. I m 32, originally from Austria, and currently living in the UK. I ve contributed to various free software projects over the years but Debian is without doubt the one I m most passionate about. I joined Debian in 2000 when I was a student. I worked on Debian more or less full time for a few years while I was pretending to study. Later I started a PhD to do research about quality and management aspects of volunteer free software projects. I investigated the release process in several free software projects, in particular looking at time-based releases. After finishing my PhD in 2007, I joined Hewlett-Packard. I m part of HP s Open Source Program Office and work on various free software and open source activities, both internally and within the community. Raphael: How did you start contributing to Debian? Martin: I first used Debian in the days of 0.93R6, some time around the end of 1995. The 0.93R6 release was still based on a.out but I needed an ELF-based system for some application, so I moved to Slackware. I then quickly moved to Red Hat Linux where I stayed for several years. I rediscovered Debian in 2000 and quickly decided to join the project. I cannot recall how I rediscovered Debian but when I did, it was clear to me that Debian was the ideal project for me: I could identify with its philosophy, I liked the volunteer nature of the project, and I found the size and diversity of Debian interesting since a large project offers a lot of different challenges and opportunities. I remember how many new things there were to learn and back then the documentation and other resources for new contributors were nowhere as good as they are today. My application manager, Julian Gilbey, was a great help he was incredibly friendly and passionate about Debian. I also remember meeting up with Peter Palfrader (weasel) for key signing when we were both in the New Maintainer queue. I was incredibly lucky with my New Maintainer process and soon became an official Debian Developer. Because there was a shortage of application managers, my first major contribution in Debian was to become an application manager myself and help other people join the project. Debian is a large project with a long history and a rich culture, so new contributors should expect that it will take some time to become familiar with everything. Fortunately, there are many resources, such as documentation and the debian-mentors list, for new contributors. Another great way to become familiar with the way things are done in Debian is to subscribe to various Debian mailing lists and ideally to read some mailing list archives. It s also a great idea to attend the Debian Conference or other conferences since meeting people in real life is a great way to integrate. I remember attending Debian Conference 1 in Bordeaux where I gave my first public talk. Finally, new contributors should find an area where they can make a unique contribution. Most people in Debian maintain packages but there are so many other ways to contribute. For example, most of my contributions were not technical but were about coordination and other organizational activities. Raphael: What s your biggest achievement within Debian? Martin: I m particularly proud of a number of achievements: Raphael: Speaking about NAS devices: what exactly are you doing on this topic and how can people help? Martin: There are plenty of instructions on the Internet to install Linux distributions on NAS or various embedded devices by connecting a serial console and then typing in hundreds of commands. What I found is that such instructions significantly limit the user base because they are way too complicated for most users. There are just too many steps that can go wrong. So instead, in Debian, we provide a solution that just works: usually, you download a firmware image for your NAS device from Debian and when you upgrade you get the Debian installer. You connect to the installer via SSH and perform a normal installation. The installer knows about the device and will prepare everything for you automatically for example, it knows if the device has requirements for the partition layout and it will install the kernel where the device expects to find it; unfortunately, NAS devices are not like PCs, so the requirements are different for almost every device and therefore you need special code to support a new device. Finally, there are detailed installation guides and we provide help on our mailing lists. There are a number of technical areas for improvement. The installation could be made even easier, and it would be nice to support new platforms and devices. A bigger problem is that while we ve implemented a great solution for NAS devices, we haven t really extended this work to support other classes of devices. For example, tablets and mobile phones are getting incredibly popular and we don t have a compelling solution for such devices, mostly because of the lack of an appropriate GUI. Raphael: What are your plans for Debian Wheezy? Martin: I ve recently been asked by Stefano Zacchiroli, our current Debian Project Leader, to coordinate the care-taking of Debian finances. Debian, as a volunteer project, relies on donations and in-kind gifts (e.g. hardware) to maintain its infrastructure and to support various development efforts, such as funding sprints and other developer gatherings. Debian s money and other assets are held by affiliate organizations around the world. My responsibility will be to keep track of money and other assets (e.g. hardware and trademarks), work with the DPL to establish procedures related to the use of Debian s assets, and make sure that the procedures are followed. Finally, we want to publish public statements so our donors know how we use their donations to further improve Debian. I just started working on this and this will be my main activity in Debian in the coming months. Raphael: Speaking of money, I plan to run a fundraising to get the Debian book I wrote with Roland Mas translated (cf. http://debian-handbook.info). Is this something Debian should support? Martin: First of all, I should make it clear that I don t decide how Debian spends its money. This is up to the DPL to decide together with the project at a whole. I ll just make sure that procedures are followed and expenses tracked and reported properly. Having said that, in my opinion, it s unlikely that Debian as a project will fund this effort. It would be inconsistent with the position of the project not to fund work directly (only some related expenses, such as travel costs to allow Debian teams to organize face-to-face meetings). Whether Debian should support the fundraising effort by helping to promote it is another question and that s probably not as clear cut. It looks like a worthwhile effort, but on the other hand it would be unfair for authors of other Debian books for Debian to put its weight behind one and there are many other efforts that are worth promoting if you promote one, where do you stop? So while it sounds worthwhile, it s probably better for Debian to stay out of it. But somehow related to this, I sometimes worry about the fact that there are so few paid opportunities around Debian. If you contribute to the Linux kernel for a while, you have an excellent chance to get hired by someone and to work on the kernel full time. The kernel may be an extreme example but there are a lot of projects that have more paid opportunities than Debian, e.g. Mono, GNOME, OpenOffice/LibreOffice and KDE. Obviously, there are some Debian Developers who can spend some time on Debian as part of their job. I know that some Canonical employees contribute to Debian, that support companies like credativ improve Debian as part of their work, and that system administrators fix bugs or package new software as they deploy Debian. But I think this is a minority of contributors and even they don t work full time on Debian. Instead what I see is that a lot of people leave university, get a job and then no longer have time for Debian or people start a family and no longer have time. I can take myself as an example since I don t have nearly as much time as I did in the past when I was a student. I guess there are different ways to deal with this problem one would be to create more paid opportunities around Debian outside the project, another one might be to make it easier for new volunteers to join the project. I don t have the answers to these questions but it s something I wonder about, and I also wonder whether pure volunteer projects can still keep up with projects with a lot of full time contributors. Raphael: What motivates you to continue to contribute year after year? Martin: Debian is a great project with a great mission, goals and people. I contribute to make Debian a better solution and to promote the free software philosophy. Finally, the community around Debian provides a lot of motivation. It s amazing how much I ve learned about other cultures because of my involvement in Debian and how many friends I ve made over the years all around the world. Raphael: Do you have wishes for Debian Wheezy? Martin: Not really. I m pretty happy with the way things are going at the moment. We have made a lot of organizational changes in the last few years from which the project has greatly benefited. I m particularly pleased about the plans to adopt a time-based freeze. Raphael: Is there someone in Debian that you admire for their contributions? Martin: There are many people I admire greatly. I d like to mention Joey Hess because he s a great example to follow. He doesn t get involved in politics, is easy to work with and does great technical work. In fact, he has made not one but several contributions that have completely changed Debian (debconf, debhelper, and debian-installer). Furthermore, Debian has a lot of contributors who have done great work over the years but who are not very vocal about it. People like Colin Watson or Peter Palfrader. Debian has many unique contributors and the list of people I admire is much longer than the few people I just mentioned.
Thank you to Martin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. He raised some interesting questions. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.